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Abstract 

We propose a theory to describe fermion mixing. The theory respects the maximal 
abelian family symmetries which are spontaneously broken down at a large scale. We 
find that quark mixing can be well described in this theory. Some concrete models are 
constructed. The crucial point for model building is to introduce some new particles 
which are SU(2) gauge singlets and carry -1/3 electric charge. The approximate family 
symmetries in our low energy effective theory are naturally achieved. Some interesting 
phenomena in the effective theory, especially with two Higgs-doublets, are discussed. 



In the standard model (SM) since all the Yukawa couplings are taken as free parameters, 
it lacks of an explanation on the pattern of the fermion masses and mixing, i.e. all the 
fermion masses and Kobayashi-Maskawa (KM) matrix are inputs of the model. This is 
usually thought as a shortcoming of the SM because of too many parameters. Moreover, 
though gauge interaction respects a flavor symmetry G = U (3)g x U (3)u x U(3)jj x U(3)l x 
£7(3) e acting on quarks and leptons, where Q and L are SU(2)-doublet quarks and leptons, 
U, D, E are singlet quarks and leptons respectively, the non-vanishing Yukawa couplings 
break this symmetry explicitly. Therefore it seems that in SM there exists a special flavor 
basis, we call it physical flavor basis. This basis is only corresponding to a set of particular 
form of Yukawa couplings, which would generate correct fermion masses and mixing. Then 
this arises a question of how to define a physical flavor basis of fermion in SM. We think 
there are generally two possible ways. One may postulate a particular form of Yukawa 
coupling, an example is democracy mechanism [|lj, only the basis related to this particular 
form of Yukawa coupling is physical flavor basis, since in fact changing a basis through 
G transformation generally changes the form of Yukawa coupling too. Another way is to 
impose a certain family symmetry on fermions. A physical flavor basis can be singled out 
depending on the assignment of quantum number of the symmetry group. In this case 
the physical basis is specified first, while the pattern of fermion masses and mixing are 
determined only after the symmetry breaking. 

The family symmetry group could be abelian or non-abelian. For three families SU(2) 
and SU(3) are the usual choices for non-abelian symmetry group 0. In this work we would 
like to look into the abelian case. We use F, S and T to denote the first, second and third 
family. Then the maximal abelian flavor symmetry group we consider is U(1)f x U(1)s X 
U(1)t acting on quarks and leptons ||, we will call them G q and G[ respectively. The 
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Yukawa interaction respecting this symmetry can be written as 

L Y = faQaU a $ + f' a Q a D a $ + g a l a E a $ + h.c. (1) 

Here a is family index, $ is standard Higgs-doublet. If the symmetry is not broken Yukawa 
couplings, therefore fermion masses, are all diagonal. So there is no mixing among different 
generations at this stage. Of course the reason for this is that different generations are 
assigned to different kind of conserved charges. To obtain non-trivial quark mixing, which 
is necessary to explain the observed KM matrix, the family symmetry must be spontaneously 
broken down at some scale V. The purpose of this work is to construct a theory, we will call 
it a underlying theory, which can realize the correct quark mixing after the flavor symmetry 
breaking. 

We believe there are many possibilities to construct a underlying theory which can 
give rise to the correct results on fermion masses and mixing. But here we constrain ourself 
within the following conditions as our criteria to build our underlying theory, (i) we demand 
V is larger than weak-scale v, namely SM could be viewed as a low energy effective theory 
of the underlying theory. And besides weak-scale v, V is the only physical scale below grand 
unification (GUT) scale in the theory, (ii) There exists a hierarchy on tree level Yukawa 
couplings, / 3 >> / 2 >> /i, similar relations are true for other terms. This requirement 
simply reflects the fact of fermion mass hierarchy. So the third family is defined as the 
heaviest one. (iii) Off-diagonal elements of the Yukawa interaction are induced only at 
loop level. This is probably true for quarks and charged leptons but not necessarily true 
for neutrinos in all the cases. As a result of this ansatz, one can easily have an order of 
magnitude explanation on the quark mixing as we will see below. 

With these assumptions, even without a concrete model, one is able to obtain some 
qualitative features of the fermion mixing. After spontaneous family symmetry breaking, 
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the Yukawa couplings are no longer diagonal. The non-diagonal couplings are induced due 
to symmetry breaking effects. In principle there are infinite number of operators induced 
in low energy effective interaction, but the dominant operators are those with the lowest 
dimension due to the fact that V is larger than weak scale v. Those lowest dimension 
operators, which must be SU(2) xU(l) gauge invariant, can be written as 

L e Y ff = L Y + UQaU b * + CbQaD^ + VabL a E b <Z> + ii ab L T a L 6 $ T $ + h.c. (2) 

Here a, b are family indices. L T a L h combines to SU(2)-singlet or triplet. 

The off-diagonal elements of the couplings £ a ft, £' ab and r\ ab are responsible for quark and 

charged lepton mixing. The fourth term is mostly responsible for generation of neutrino 

mass and mixing. From dimension analysis and the assumption (iii), we estimate that 

Cab, Cab( a b) ~ — hjA a 5. £ and £' are dimensionless, and 1/167T 2 is the loop factor. / 3 is 
1 (in 

the Yukawa coupling of the heaviest quark. For t quark as heavy as 170 GeV, / 3 should be 

order of one. Of course this is only a rough estimation. One order of magnitude larger or 

smaller should be reasonable. All uncertainties and other effects can be absorbed in \ a b. As 

9s 

for the leptons there is a similar result, i.e. r] ab (a ^ b) ~ -X' b but with g 3 ~ 1CT 2 , for 

16n 2 

tau mass is around 1.7 GeV. However, one should also notice the difference between leptons 
and quarks at this point. For example in SM quark mixing is non-trivial, but the lepton 
mixing is trivial due to absence of right-handed neutrino. In a more complicated model, 
as we will see below, lepton mixing can occur generally. The fourth term is a dimension 
five operator and it breaks lepton number explicitly by two units. If lepton number is not 

broken then rj' b = 0, otherwise r]' ab ~ ^rr^ab- This should be a natural result since we 

expect a small neutrino mass for large V. 

We denote the upper and down quarks, charged lepton and neutrino mass matrixes as 
M u , M D , M L and M v respectively. The diagonal elements of these matrixes are inputs in 
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our theory. In practice we choose Mg ~ 170 GeV, Mg ~ 1.5 GeV>> Mft; ~ 5 GeV, 
~ 0.2 GeV» Af£ and M 3 L 3 ~ 1.7 GeV, M 2 L 2 ~ 0.1 GeV» Af£; The mass matrixes 
are diagonalized by a set of bi-unitarity transformations on left, right-handed quarks and 
leptons. If U and V are the unitarity matrixes acting on ^ and D L , the KM matrix is 
defined as K = U + V. With a not unreasonable choice of parameters: A 32 /167r 2 ~ 10~ 3 and 
A 32 ~ 6A 3 i ~ 6A 2 i, the matrix elements of U and V are evaluated as 

U 32 ~ Mg/Mg ~ QMg/Mg 3 ~ 6C/ 31 ~ 10- 3 U 2 i ~ Mg/Mg ~ 2 x 10~ 2 

(3) 

V 32 ~ Mg/Mg ~ 6M 3 ^/M3^ ~ 6F 3 i ~3x 10" 2 V21 ~ M 2 ^/M 2 ^ ~ 0.2 
From this one can see that down quark mixing is larger than the mixing of up quark, and 
mixing between the third and the first two generations is smaller than mixing of the first 
two generations. This results in correct KM matrix elements at a factor of 2 or 3 level. And 
in this theory one can easily understand why K 32 is much smaller than K 2 \. In fact all the 
results are due to the fact that the third generation is much heavier than other two, up 
quarks are heavier than the corresponding down quarks and the non-diagonal elements of 
mass matrixes are only generated at one loop level. One interesting observation is that the 
loop effects which induce off-diagonal Yukawa interaction for quarks and charged leptons 
could be independent of the flavor symmetry breaking scale. Therefore one is free to push 
V very high to satisfy other requirements. In contradiction with this is the neutrino mass, 
which is inversely proportional to the scale V, namely m v ~ 10~ 4 A'f 2 /l / GeV. To get 
neutrino mass smaller than a few eV, V must be larger than 10 7 — 10 8 GeV, if A' is not very 
small, i.e. A' ~ 0.1 — 1. 

Spontaneous breaking of the global family symmetries will result in some Nambu- 
Goldston bosons. These bosons couple to quarks, leptons or both. There exists strong 
constrains on these couplings from cosmological and astrophysical considerations. Gener- 
ally the constrains imply a lower limit on the scale of symmetry breaking V. In our theory 
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since these couplings are induced only from loop diagrams. Hence the lower limit on V is 
about two order of magnitudes smaller than in other cases. For quark- Goldston coupling 
the strongest constrains are from SN1987a |4[], which imply a lower bound V > 10 8 GeV. For 
lepton-Goldston coupling the most stringent restriction is due to the cooling of red giants 
0. It requires that the scale V must be larger than about 10 7 GeV. 

We now consider explicit models which yield this low-energy effective theory. 

Model I: To extend SM we introduce a group of scalar particles H a b which are SU(2)- 
singlets and carry -1/3 electric charge. Under SU(3) color transformation, they behave like 
3 or 6 plets. Scalars with these properties will couple to quarks as following: 

L' Y = 9abQaQbH ab + g' ab UjD b H ab + h.c. (4) 

In order to break the family symmetry spontaneously, we introduce another set of color and 
electric neutral particles S a which transform as singlet of SU(2) gauge group. The most 
general Lagrangian of the model can be written as 

L = L G + Ly + L' Y + V P (S,H,$) (5) 

where L G is gauge interaction. V is the most general potential of the scalar particles. 
Especially a term like \ ac H^ b H bc S*S c is included in V P . In this model because the particles 
H and S do not couple to leptons. We expect the lepton sector is same as that in SM. The 
family symmetry on lepton sector Gi = U(l) e x U(l)^ x U(1) T is not broken. As a result, 
Vab = Vab = ( see ec L u - (2))- At scale V, when S fields get nonzero vacuum expectation 
values, the family symmetry G q (referring to quark fields only) is broken down to U(1)b, 
where B is baryon number. The quark mixing among different generations are generated at 
one loop level as shown in Fig. 1. It is straightforward to obtain 



Zab — -T^9ac A abUc9bc + JbQbc 2 

Lb - T^9ac X ab{fcg bc + } b 9h 



(6) 



Here m# is the mass of H particle and it is understood one should sum over index c. With 
< S >~ rriH ~ V and all the elements of g and g' at same order, we get an approximate 
form for £ and £' as 



iab ~ Cfc - -jri^b (7) 
107T Z 



which is the form we discussed in previous section. And obviously they are independent of 
the scale V. 

Model II: We add another Higgs-doublet in model I. This is a two- Higgs-doublets 
model with abelian family symmetry G q spontaneously broken down to U(1)b- Gi is still 
unbroken in this model. So we will only consider quark sector. The low energy effective 
Yukawa interaction after family symmetry breaking has the form 

L Y = Q(F 1 ^ 1 + F 2 $ 2 )U + Q(Fl$! + F& 2 )D + h.c. (8) 

Still we impose the hierarchy condition ( i.e. condition (ii) ) on the diagonal elements of both 
Fi(F[) and ^(i^) in order to obtain the pattern of quark masses naturally. The description 
of quark mixing is similar as in model one. Since typically the off-diagonal elements are 
about 0.2 ~ 0.01 of the related diagonal elements. We actually obtain a two-Higgs-doublets 
model at low energy with an approximate family symmetry G q . One interesting consequence 
of the approximate symmetry is that one is able to avoid too large flavor changing neutral 
current (FCNC) naturally without resorting to Glashow and Weinberg's discrete symmetry 
||. Spontaneous CP violation is also possible in this model 0. CP violation phenomena 
of this model was discussed recently by Wu and Wolfenstein H, where the approximate 
flavor symmetry is assumed from beginning. The main results of their research are the e' 
value originated from scalar interaction is larger than in generic superweak model and in 
certain cases e'/e could be as large as 10~ 3 . The electric dipole moments of electron |§ 



and neutron [10| are close to current experimental bounds in this kind of models. Here 
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we would like to point out a very interesting prediction of D° — D° mixing in our model. 
Since the mixing matrices for both down and up quarks are determined approximately, we 
can estimate the D° — D° mixing due to tree level FCNC. Our estimation gives Am^ ~ 
10~ 14 — 10~ 13 (100GeK/mjy) 2 GeV, which is very close to present experimental bound [[U] 



while the tree level FCNC contribution to K°—K° mixing is close but below the experimental 
value. This is probably the best place to discover this new physics in future experiments. 

Model III: one group of SU(2)-singlet particles h ao are introduced based on model 
II. h a b carry one units of electric charge. These particles couple to lepton sector and the 
Higgs-doublets as following 

L le P ton = CabL T LbKb + C ' ab ^ 2 h ab S ab (9) 



Just for the lepton sector this model is well-known as Zee- type model [p!2[ . The lepton 
sector family symmetries Gi are spontaneously broken at scale V. G q and Gi breaking are 
triggered by same S fields, therefore they should be spontaneously broken at same scale. 



And the lepton number violation in the model implies neutrino will get non-zero mass. This 

_93_ 
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of charged leptons. We checked all possible flavor changing processes in lepton sector and 



means nonzero £ and £' values, £ ~ £' ~ — 2 A. In addition, there exists non-trivial mixing 



found that those flavor changing scalar interaction will not induce direct observable effects in 
the current and future experiments. Most phenomena of Zee-type model has been discussed 



extensively in literatures | 13fl . Here we just point out that in our model leptons and quarks 
are actually treated in a similar way, i.e. we introduce H particles to couple quark sector 
and at the same time h particles to lepton sector. 

In summary, we have examined the theory with U(l) 3 family symmetry which is spon- 
taneously broken down at a large energy scale. We found that KM matrix can be well 
reproduced. And at the same time an approximate abelian family symmetry at low en- 
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ergy is automatically achieved. We constructed some explicit models by introduction of the 
particles H a b in quark sector. Analogy with this is the Zee-scalar particles h a b which was 
introduced in lepton sector to generate neutrino mass. The quark mixing could be indepen- 
dent of the family symmetry breaking scale and the mass of H particle. Constrains from 
cosmology and astrophysics, and small neutrino masses imply the flavor symmetry breaking 
scale should be larger than 10 7 ~ 10 8 GeV. In models with two-Higgs doublets ( model II 
and III ) D° — D° mixing is expected to be large and very close to current experimental 
bound §. 

Finally some further developments which we believe could be interesting are in order. (1) 
In the framework of this theory, we don't have any idea on the mass hierarchy of fermions. 
A larger unification symmetry of flavor is probably needed. For instance, it is already 
known that in some models with SU(3) flavor symmetry, one generation fermion could be 



"naturally" much heavier than another two WZ\. One interesting possibility is to consider 



flavor symmetry breaking in two steps. At the first step SU(3) symmetry is broken down 
to U(l) 3 . This possibility is now under investigation. (2) It is very desired to incorporate 
the models in GUT theory, h particles can occur through 10 plets of SU(5). H particles can 
be the number of, for example, 45 plets of Higgs fields in SU(5). Though as we said we can 
deal with quarks and leptons in the similar way as in model III, it is not in any sense an 
unification theory. A grand unification theory with certain flavor symmetry probably will 
offer us a deeper understanding of the pattern of fermion mass and mixing. 
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